A retrospective study using thousands of two-dimensional mammograms performed at Kaiser Permanente Northern California showed that five separate AI algorithms outperformed the standard clinical model of predicting five-year risk for breast cancer.
Women’s risk for developing breast cancer uses clinical models such as the Breast Cancer Surveillance Consortium (BCSC) risk model. BCSC uses self-reported information combined with other patient data such as age, family history of breast cancer, whether a woman has given birth, and whether a woman has dense breasts to calculate a risk score.
But these methods have some shortcomings, noted Vignesh A. Arasu, MD, PhD, a research scientist and practicing radiologist at Kaiser Permanente Northern California.
“Clinical risk models depend on gathering information from different sources, which isn’t always available or collected,” said Arasu, lead researcher of the study, which appears today in the journal Radiology. “Recent advances in AI deep learning provide us with the ability to extract hundreds to thousands of additional mammographic features.”
For this study, Arasu and the team used data from screening 2D mammograms performed at Kaiser Permanente Northern California in 2016. From a collection of more than 324,000 women who met eligibility criteria, a sub-cohort of 13,628 were randomly selected for analysis. In addition, 4,584 patients who were subsequently diagnosed with cancer within five years of their mammogram were also studied, with all women followed for five years. Using the entire year of screenings allowed provided a representative sample of the communities in Northern California served by the health system.
Using the mammograms, risk scores for cancer over the five-year period were generated by five separate AI risk-prediction algorithms—two used by researchers in academia, and three commercially available algorithms. The investigators divided the study into three time periods: interval cancer risk, or incident cancers diagnosed up to one year after screening; future cancer risk, or incident cancers diagnosed from between one and five years; and all cancer risk, or incident cancers diagnosed between initial screening and five years.
The risk scores of the five algorithms were then compared with the standard BCSC clinical risk score.
“All five AI algorithms performed better than the BCSC risk model for predicting breast cancer risk at zero to five years,” Arasu said. “This strong predictive performance over the five-year period suggests AI is identifying both missed cancers and breast tissue features that help predict future cancer development. Something in mammograms allows us to track breast cancer risk. This is the ‘black box’ of AI.”
The study showed that some of the five algorithms were particularly good at predicting the high risk of the development of interval cancer, and important finding that can help determine whether a patient needs supplemental screening or short-interval follow up. One example of the better performance of AI is that when evaluating patients with the highest 10% risk, AI predicted 28% of cancers, while BCSC predicted 21%.
Further, the investigators said that even the AI tools that were trained to determine risk for shorter time horizons—as low as three months—were also able to predict the future risk of cancer up to five years when no cancer was shown via mammography. Used together, the AI models and BCSC risk model provided even greater accuracy in predicting future breast cancer risk.
Arasu noted that AI is already used by some radiologists to help detect cancer on mammograms, but also noted it now has the utility to generate risk scores that could be incorporated into the reporting of imaging results.
“AI for cancer risk prediction offers us the opportunity to individualize every woman’s care, which isn’t systematically available. It’s a tool that could help us provide personalized, precision medicine on a national level,” Arasu concluded.
